{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "keras.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/arbi11/CompEM1/blob/master/keras.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ht58xp7TOpQV",
        "colab_type": "text"
      },
      "source": [
        "Download and install the TensorFlow 2.0 Beta package.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YAH3a9ohNdW4",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "outputId": "00d383e5-8ba6-4024-cf16-9e3075ffaa69"
      },
      "source": [
        "! pip install -q tensorflow==2.0.0-beta1"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[K     |████████████████████████████████| 87.9MB 365kB/s \n",
            "\u001b[K     |████████████████████████████████| 501kB 39.5MB/s \n",
            "\u001b[K     |████████████████████████████████| 3.1MB 39.6MB/s \n",
            "\u001b[?25h"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5GcEndrUOsh8",
        "colab_type": "text"
      },
      "source": [
        " Import TensorFlow into your program:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sPODg1zDOjAP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import tensorflow as tf"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F0HaqI9ePGrM",
        "colab_type": "text"
      },
      "source": [
        "Tensorflow 2.0\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "All the code examples should work fine with tf.keras, the only difference is how to import Keras:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DpY5Vm-qOvD8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras import Sequential, layers\n",
        "from tensorflow.keras.layers import Dense, Flatten"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qaoil82HPMR3",
        "colab_type": "text"
      },
      "source": [
        "Build the `tf.keras.Sequential` model by stacking layers. Choose an optimizer and loss function for training:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DLW1beHUPLp6",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "2c451fd0-60f1-4982-fed1-077d747d6984"
      },
      "source": [
        "print(tf.__version__)\n",
        "print(tf.keras.__version__)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2.0.0-beta1\n",
            "2.2.4-tf\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7233qkIBQMuE",
        "colab_type": "text"
      },
      "source": [
        "[fully connected NN](https://www.oreilly.com/library/view/tensorflow-for-deep/9781491980446/assets/tfdl_0402.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qtSgylEPVtZv",
        "colab_type": "text"
      },
      "source": [
        "Getting the data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dj7nJhClVvv6",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        },
        "outputId": "2bc9cf4a-29de-439b-ecda-dfce3b47335c"
      },
      "source": [
        "! git clone https://github.com/arbi11/CompEM1.git"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Cloning into 'CompEM1'...\n",
            "remote: Enumerating objects: 36, done.\u001b[K\n",
            "remote: Counting objects: 100% (36/36), done.\u001b[K\n",
            "remote: Compressing objects: 100% (36/36), done.\u001b[K\n",
            "remote: Total 36 (delta 12), reused 3 (delta 0), pack-reused 0\u001b[K\n",
            "Unpacking objects: 100% (36/36), done.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L_iecw75V1vt",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "86d82390-3fb5-490e-ede2-c9162c36e9db"
      },
      "source": [
        "cd CompEM1/"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/CompEM1/CompEM1/CompEM1/CompEM1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8BXMVmbSV2_6",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "outputId": "bd12451a-6f41-4a60-bbf9-6b2a8ec3031c"
      },
      "source": [
        "! ls"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1_pandas_intro.ipynb\t columns.csv\t\t   NLP.ipynb\n",
            "1_pandas_intro_RF.ipynb  data.csv\t\t   README.md\n",
            "2_RF.ipynb\t\t indian_liver_patient.csv  target.csv\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4J3xpP-GV9YV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "data_frame = pd.read_csv('data.csv')\n",
        "target_frame = pd.read_csv('target.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hPeMw4uMWEqE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "columns = ['StatorYokeThickness', 'ToothWidth', 'AirGap', 'InsetDepth', 'MagnetThickness', 'MagnetWidth']"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MioR0kFdXRKm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "X = data_frame.iloc[:, :].values\n",
        "y = target_frame.iloc[:, :].values\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gD9bCxiQYgY-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "e773fcb9-8a1a-4469-8da3-e2fb58c840b9"
      },
      "source": [
        "X.shape"
      ],
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(9999, 6)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 57
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uhKJMNnkYiTc",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "b18cbd02-7163-4884-cd10-984f2b9ab8b9"
      },
      "source": [
        "y.shape"
      ],
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(9999, 1)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 58
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lqxwl0ULPVjK",
        "colab_type": "text"
      },
      "source": [
        "# 2 **Models types**\n",
        "\n",
        "Sequential\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "\n",
        "Simple stack of layers\n",
        "\n",
        "\n",
        "\n",
        "Functional\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "Multi input, multi output, shared layers, non sequential flows\n",
        "\n",
        "\n",
        "\n",
        "Model Subclassing\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "Customizable, define foward pass with call method. Enables eager execution\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UKMaxJsdSU-Z",
        "colab_type": "text"
      },
      "source": [
        "### 2.1 Sequential model\n",
        "\n",
        "Building the neural network requires configuring the layers of the model, then compiling the model.\n",
        "\n",
        "Much of deep learning consists of stacking together simple layers."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BGd_uXqrPmrN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# maximum simplicity\n",
        "seq_model = Sequential()\n",
        "seq_model.add(Dense(20, activation='relu', input_shape=(6,)))\n",
        "seq_model.add(Dense(20, activation='relu'))\n",
        "seq_model.add(Dense(1, activation='sigmoid'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ehSQyJ_1SEtJ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 272
        },
        "outputId": "5b11549b-99cc-45f4-e808-cf6aefda0ee4"
      },
      "source": [
        "print(seq_model.summary())"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_3 (Dense)              (None, 20)                140       \n",
            "_________________________________________________________________\n",
            "dense_4 (Dense)              (None, 20)                420       \n",
            "_________________________________________________________________\n",
            "dense_5 (Dense)              (None, 1)                 21        \n",
            "=================================================================\n",
            "Total params: 581\n",
            "Trainable params: 581\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "None\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "06LdW2-kSB8O",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "seq_model.compile(optimizer='adam',\n",
        "              loss='mse',\n",
        "              metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AQ6DhbLAR-KX",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 391
        },
        "outputId": "1e5f8eda-c6a8-47ed-a379-ea7f909d66f3"
      },
      "source": [
        "seq_model.fit(X, y, epochs=10, batch_size=32)"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 9999 samples\n",
            "Epoch 1/10\n",
            "9999/9999 [==============================] - 1s 54us/sample - loss: 0.0040 - accuracy: 0.0000e+00\n",
            "Epoch 2/10\n",
            "9999/9999 [==============================] - 0s 40us/sample - loss: 0.0012 - accuracy: 0.0000e+00\n",
            "Epoch 3/10\n",
            "9999/9999 [==============================] - 0s 40us/sample - loss: 7.8012e-04 - accuracy: 0.0000e+00\n",
            "Epoch 4/10\n",
            "9999/9999 [==============================] - 0s 42us/sample - loss: 5.3863e-04 - accuracy: 0.0000e+00\n",
            "Epoch 5/10\n",
            "9999/9999 [==============================] - 0s 40us/sample - loss: 4.7443e-04 - accuracy: 0.0000e+00\n",
            "Epoch 6/10\n",
            "9999/9999 [==============================] - 0s 39us/sample - loss: 4.7128e-04 - accuracy: 0.0000e+00\n",
            "Epoch 7/10\n",
            "9999/9999 [==============================] - 0s 39us/sample - loss: 4.0453e-04 - accuracy: 0.0000e+00\n",
            "Epoch 8/10\n",
            "9999/9999 [==============================] - 0s 38us/sample - loss: 3.7281e-04 - accuracy: 0.0000e+00\n",
            "Epoch 9/10\n",
            "9999/9999 [==============================] - 0s 39us/sample - loss: 3.6107e-04 - accuracy: 0.0000e+00\n",
            "Epoch 10/10\n",
            "9999/9999 [==============================] - 0s 39us/sample - loss: 3.1551e-04 - accuracy: 0.0000e+00\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7fa6cd227e48>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WlYECFo6Y4NK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "seq_model.compile(optimizer='adam',\n",
        "              loss='mse',              \n",
        "              metrics=[keras.metrics.mean_squared_error, \n",
        "                       keras.metrics.mean_absolute_error, \n",
        "                       keras.metrics.mean_absolute_percentage_error\n",
        "                       ]\n",
        "              )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZOZjFg9cajMO",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 411
        },
        "outputId": "11e8b267-ff56-40cb-c8d3-8b7ba041b449"
      },
      "source": [
        "seq_model.fit(X, y, epochs=10, batch_size=32)"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 9999 samples\n",
            "Epoch 1/10\n",
            "9999/9999 [==============================] - 1s 54us/sample - loss: 3.5508e-04 - mean_squared_error: 3.5508e-04 - mean_absolute_error: 0.0125 - mean_absolute_percentage_error: 1.4917\n",
            "Epoch 2/10\n",
            "9999/9999 [==============================] - 0s 43us/sample - loss: 3.6950e-04 - mean_squared_error: 3.6950e-04 - mean_absolute_error: 0.0128 - mean_absolute_percentage_error: 1.5261\n",
            "Epoch 3/10\n",
            "9999/9999 [==============================] - 0s 40us/sample - loss: 3.7911e-04 - mean_squared_error: 3.7911e-04 - mean_absolute_error: 0.0125 - mean_absolute_percentage_error: 1.5134\n",
            "Epoch 4/10\n",
            "9999/9999 [==============================] - 0s 42us/sample - loss: 3.3293e-04 - mean_squared_error: 3.3293e-04 - mean_absolute_error: 0.0120 - mean_absolute_percentage_error: 1.4375\n",
            "Epoch 5/10\n",
            "9999/9999 [==============================] - 0s 41us/sample - loss: 3.2432e-04 - mean_squared_error: 3.2432e-04 - mean_absolute_error: 0.0119 - mean_absolute_percentage_error: 1.4247\n",
            "Epoch 6/10\n",
            "9999/9999 [==============================] - 0s 40us/sample - loss: 2.8517e-04 - mean_squared_error: 2.8517e-04 - mean_absolute_error: 0.0114 - mean_absolute_percentage_error: 1.3603\n",
            "Epoch 7/10\n",
            "9999/9999 [==============================] - 0s 42us/sample - loss: 2.6546e-04 - mean_squared_error: 2.6546e-04 - mean_absolute_error: 0.0111 - mean_absolute_percentage_error: 1.3143\n",
            "Epoch 8/10\n",
            "9999/9999 [==============================] - 0s 43us/sample - loss: 2.4822e-04 - mean_squared_error: 2.4822e-04 - mean_absolute_error: 0.0108 - mean_absolute_percentage_error: 1.2808\n",
            "Epoch 9/10\n",
            "9999/9999 [==============================] - 0s 39us/sample - loss: 2.6250e-04 - mean_squared_error: 2.6250e-04 - mean_absolute_error: 0.0109 - mean_absolute_percentage_error: 1.3007\n",
            "Epoch 10/10\n",
            "9999/9999 [==============================] - 0s 39us/sample - loss: 3.3392e-04 - mean_squared_error: 3.3392e-04 - mean_absolute_error: 0.0121 - mean_absolute_percentage_error: 1.4491\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7fa6ccefd0b8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4gUC5PCpcqsp",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import os\n",
        "local_download_path = os.path.expanduser('~/data')\n",
        "try:\n",
        "  os.makedirs(local_download_path)\n",
        "except: pass"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3vfQUOlpasOX",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        },
        "outputId": "d5ef2488-56d9-4115-f5c8-2bd556afa919"
      },
      "source": [
        "filepath = '/root/data/' + \"/weights-{epoch:02d}-{val_loss:.10f}.hdf5\"\n",
        "\n",
        "checkpoint = keras.callbacks.ModelCheckpoint(filepath, monitor='val_loss', verbose=1, \\\n",
        "                             save_best_only=False, save_weights_only=False, \\\n",
        "                             mode='auto', period=1)\n",
        "\n",
        "early_stop = keras.callbacks.EarlyStopping(monitor='val_loss', patience=5)\n",
        "\n",
        "callbacks_list = [checkpoint, early_stop]                  "
      ],
      "execution_count": 67,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "W0621 14:32:14.915827 140355446192000 callbacks.py:859] `period` argument is deprecated. Please use `save_freq` to specify the frequency in number of samples seen.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qtUnStCEceoC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "val_X = X[-100:,:]\n",
        "val_y = y[-100:, :]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FcpLrdeTftyn",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "d6b09b42-ed50-472e-eb94-d65441e3fd88"
      },
      "source": [
        "val_X.shape"
      ],
      "execution_count": 81,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(100, 6)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 81
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6CYAaWqNba9k",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 530
        },
        "outputId": "c898ef01-3544-46d7-cb8b-df53baffed54"
      },
      "source": [
        "history = seq_model.fit(X, y, epochs=100, batch_size=32,\n",
        "                       validation_data= (val_X, val_y),\n",
        "                        verbose=2,\n",
        "                       callbacks = callbacks_list)\n",
        "\n"
      ],
      "execution_count": 69,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 9999 samples, validate on 9999 samples\n",
            "Epoch 1/100\n",
            "\n",
            "Epoch 00001: saving model to /root/data//weights-01-0.0001619287.hdf5\n",
            "9999/9999 - 1s - loss: 2.0447e-04 - mean_squared_error: 2.0447e-04 - mean_absolute_error: 0.0098 - mean_absolute_percentage_error: 1.1559 - val_loss: 1.6193e-04 - val_mean_squared_error: 1.6193e-04 - val_mean_absolute_error: 0.0090 - val_mean_absolute_percentage_error: 1.0578\n",
            "Epoch 2/100\n",
            "\n",
            "Epoch 00002: saving model to /root/data//weights-02-0.0001563860.hdf5\n",
            "9999/9999 - 1s - loss: 1.9822e-04 - mean_squared_error: 1.9822e-04 - mean_absolute_error: 0.0097 - mean_absolute_percentage_error: 1.1457 - val_loss: 1.5639e-04 - val_mean_squared_error: 1.5639e-04 - val_mean_absolute_error: 0.0091 - val_mean_absolute_percentage_error: 1.0597\n",
            "Epoch 3/100\n",
            "\n",
            "Epoch 00003: saving model to /root/data//weights-03-0.0001713192.hdf5\n",
            "9999/9999 - 1s - loss: 1.9336e-04 - mean_squared_error: 1.9336e-04 - mean_absolute_error: 0.0096 - mean_absolute_percentage_error: 1.1331 - val_loss: 1.7132e-04 - val_mean_squared_error: 1.7132e-04 - val_mean_absolute_error: 0.0092 - val_mean_absolute_percentage_error: 1.0914\n",
            "Epoch 4/100\n",
            "\n",
            "Epoch 00004: saving model to /root/data//weights-04-0.0001575110.hdf5\n",
            "9999/9999 - 1s - loss: 2.0494e-04 - mean_squared_error: 2.0494e-04 - mean_absolute_error: 0.0100 - mean_absolute_percentage_error: 1.1785 - val_loss: 1.5751e-04 - val_mean_squared_error: 1.5751e-04 - val_mean_absolute_error: 0.0086 - val_mean_absolute_percentage_error: 1.0179\n",
            "Epoch 5/100\n",
            "\n",
            "Epoch 00005: saving model to /root/data//weights-05-0.0003133468.hdf5\n",
            "9999/9999 - 1s - loss: 2.1884e-04 - mean_squared_error: 2.1884e-04 - mean_absolute_error: 0.0100 - mean_absolute_percentage_error: 1.1840 - val_loss: 3.1335e-04 - val_mean_squared_error: 3.1335e-04 - val_mean_absolute_error: 0.0139 - val_mean_absolute_percentage_error: 1.5902\n",
            "Epoch 6/100\n",
            "\n",
            "Epoch 00006: saving model to /root/data//weights-06-0.0001600571.hdf5\n",
            "9999/9999 - 1s - loss: 1.8945e-04 - mean_squared_error: 1.8945e-04 - mean_absolute_error: 0.0096 - mean_absolute_percentage_error: 1.1326 - val_loss: 1.6006e-04 - val_mean_squared_error: 1.6006e-04 - val_mean_absolute_error: 0.0087 - val_mean_absolute_percentage_error: 1.0236\n",
            "Epoch 7/100\n",
            "\n",
            "Epoch 00007: saving model to /root/data//weights-07-0.0001768738.hdf5\n",
            "9999/9999 - 1s - loss: 1.8388e-04 - mean_squared_error: 1.8388e-04 - mean_absolute_error: 0.0094 - mean_absolute_percentage_error: 1.1077 - val_loss: 1.7687e-04 - val_mean_squared_error: 1.7687e-04 - val_mean_absolute_error: 0.0091 - val_mean_absolute_percentage_error: 1.0655\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q2cdGMGUb0HQ",
        "colab_type": "text"
      },
      "source": [
        "hist = model.fit(generator = training_generator,\n",
        "                    steps_per_epoch = list_IDs_train//batch_size,\n",
        "                    validation_data = validation_generator,\n",
        "                    validation_steps = list_IDs_val//batch_size,\n",
        "                    epochs= epochs,\n",
        "                    verbose=1,\n",
        "                    callbacks= callbacks_list\n",
        "                    )"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ginc33ejP13-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model = tf.keras.models.Sequential([\n",
        "  tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
        "  tf.keras.layers.Dense(128, activation='relu'),\n",
        "  tf.keras.layers.Dropout(0.2),\n",
        "  tf.keras.layers.Dense(10, activation='softmax')\n",
        "])\n",
        "\n",
        "model.compile(optimizer='adam',\n",
        "              loss='sparse_categorical_crossentropy',\n",
        "              metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AxcJ-udBUGGA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KQhkaZ28mTA9",
        "colab_type": "text"
      },
      "source": [
        "### 2.2 Functional Model\n",
        "[example](https://cdn-images-1.medium.com/max/2600/1*6hF97Upuqg_LdsqWY6n_wg.png)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D2vLlyACP7w1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# more full featured and flexible\n",
        "inputs = keras.Input(shape=(6,))\n",
        "x = Dense(20, activation='relu')(inputs)\n",
        "x = Dense(20, activation='relu')(x)\n",
        "outputs = Dense(1, activation='sigmoid')(x)\n",
        "\n",
        "func_model = keras.Model(inputs, outputs)\n",
        "func_model.compile(optimizer='adam',\n",
        "              loss='mse',\n",
        "              metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E6dthaEKguOl",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 306
        },
        "outputId": "aced87bc-a6e7-4bdf-8e6d-138b235c7704"
      },
      "source": [
        "print(func_model.summary())"
      ],
      "execution_count": 105,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"model_33\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "input_36 (InputLayer)        [(None, 6)]               0         \n",
            "_________________________________________________________________\n",
            "dense_62 (Dense)             (None, 20)                140       \n",
            "_________________________________________________________________\n",
            "dense_63 (Dense)             (None, 20)                420       \n",
            "_________________________________________________________________\n",
            "dense_64 (Dense)             (None, 1)                 21        \n",
            "=================================================================\n",
            "Total params: 581\n",
            "Trainable params: 581\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "None\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "79kPIl_afOtZ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 391
        },
        "outputId": "165e654e-7483-4f32-f685-d6d02a833adc"
      },
      "source": [
        "func_model.fit(X, y, epochs=10, batch_size=32)"
      ],
      "execution_count": 106,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 9999 samples\n",
            "Epoch 1/10\n",
            "9999/9999 [==============================] - 0s 47us/sample - loss: 0.1122 - accuracy: 0.0000e+00\n",
            "Epoch 2/10\n",
            "9999/9999 [==============================] - 0s 38us/sample - loss: 0.0125 - accuracy: 0.0000e+00\n",
            "Epoch 3/10\n",
            "9999/9999 [==============================] - 0s 38us/sample - loss: 0.0125 - accuracy: 0.0000e+00\n",
            "Epoch 4/10\n",
            "9999/9999 [==============================] - 0s 38us/sample - loss: 0.0125 - accuracy: 0.0000e+00\n",
            "Epoch 5/10\n",
            "9999/9999 [==============================] - 0s 38us/sample - loss: 0.0125 - accuracy: 0.0000e+00\n",
            "Epoch 6/10\n",
            "9999/9999 [==============================] - 0s 39us/sample - loss: 0.0089 - accuracy: 0.0000e+00\n",
            "Epoch 7/10\n",
            "9999/9999 [==============================] - 0s 37us/sample - loss: 0.0027 - accuracy: 0.0000e+00\n",
            "Epoch 8/10\n",
            "9999/9999 [==============================] - 0s 38us/sample - loss: 0.0020 - accuracy: 0.0000e+00\n",
            "Epoch 9/10\n",
            "9999/9999 [==============================] - 0s 39us/sample - loss: 0.0013 - accuracy: 0.0000e+00\n",
            "Epoch 10/10\n",
            "9999/9999 [==============================] - 0s 39us/sample - loss: 0.0011 - accuracy: 0.0000e+00\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7fa6cb8cc2e8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 106
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JudrwxmPfOAj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "func_model.save('/root/data/func_model.h5')\n",
        "\n",
        "model = tf.keras.models.load_model('/root/data/my_model.h5')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wXkMKDI7QCQf",
        "colab_type": "text"
      },
      "source": [
        "### 2.3 Model Subclassing"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CmS67rDrP8xu",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 391
        },
        "outputId": "7ef93127-7386-4fdb-f3c8-47be18652f13"
      },
      "source": [
        "# maximally flexible and hackable\n",
        "class ScModel(keras.Model):\n",
        "  \n",
        "  def __init__(self):\n",
        "    super(ScModel, self).__init__()\n",
        "    self.dense1 = Dense(20, activation='relu')\n",
        "    self.dense2 = Dense(20, activation='relu')\n",
        "    self.dense3 = Dense(1, activation='sigmoid')\n",
        "    \n",
        "  def call(self, inputs):\n",
        "    x = self.dense1(inputs)\n",
        "    x = self.dense2(x)\n",
        "    return self.dense3(x)\n",
        "  \n",
        "sc_model = ScModel()\n",
        "sc_model.compile(optimizer='adam',\n",
        "              loss='mse',\n",
        "              metrics=['accuracy'])\n",
        "sc_model.fit(X, y, epochs=10, batch_size=32)"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 9999 samples\n",
            "Epoch 1/10\n",
            "9999/9999 [==============================] - 0s 49us/sample - loss: 0.0033 - accuracy: 0.0000e+00\n",
            "Epoch 2/10\n",
            "9999/9999 [==============================] - 0s 38us/sample - loss: 0.0014 - accuracy: 0.0000e+00\n",
            "Epoch 3/10\n",
            "9999/9999 [==============================] - 0s 38us/sample - loss: 8.5259e-04 - accuracy: 0.0000e+00\n",
            "Epoch 4/10\n",
            "9999/9999 [==============================] - 0s 38us/sample - loss: 5.5859e-04 - accuracy: 0.0000e+00\n",
            "Epoch 5/10\n",
            "9999/9999 [==============================] - 0s 37us/sample - loss: 4.8726e-04 - accuracy: 0.0000e+00\n",
            "Epoch 6/10\n",
            "9999/9999 [==============================] - 0s 37us/sample - loss: 4.3587e-04 - accuracy: 0.0000e+00\n",
            "Epoch 7/10\n",
            "9999/9999 [==============================] - 0s 36us/sample - loss: 3.9802e-04 - accuracy: 0.0000e+00\n",
            "Epoch 8/10\n",
            "9999/9999 [==============================] - 0s 37us/sample - loss: 3.5547e-04 - accuracy: 0.0000e+00\n",
            "Epoch 9/10\n",
            "9999/9999 [==============================] - 0s 38us/sample - loss: 3.3689e-04 - accuracy: 0.0000e+00\n",
            "Epoch 10/10\n",
            "9999/9999 [==============================] - 0s 36us/sample - loss: 3.3520e-04 - accuracy: 0.0000e+00\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7fa6cc5aa748>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YanP1bHrQ4c7",
        "colab_type": "text"
      },
      "source": [
        "### Visualize Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S-vDD0lUQNx9",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        },
        "outputId": "ed22d442-b131-4c19-983b-fad6a81e01ba"
      },
      "source": [
        "sc_model.summary()"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sc_model\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_9 (Dense)              multiple                  140       \n",
            "_________________________________________________________________\n",
            "dense_10 (Dense)             multiple                  420       \n",
            "_________________________________________________________________\n",
            "dense_11 (Dense)             multiple                  21        \n",
            "=================================================================\n",
            "Total params: 581\n",
            "Trainable params: 581\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TorgcTHNZVxq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}